{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CIFAR10 classification using CNN codes\n",
    "\n",
    "Here we are going to build linear models to classify CNN codes of CIFAR10 images.\n",
    "\n",
    "We assume that we already have all the codes extracted by the scripts in the following notebooks:\n",
    "- [Feature_extraction_using_keras.ipynb](Feature_extraction_using_keras.ipynb)\n",
    "- [Feature_extraction_using_Inception_v3.ipynb](Feature_extraction_using_Inception_v3.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CIFAR10_Inception_v3_features.npz    CIFAR10_vgg16-keras_features.npz\r\n",
      "CIFAR10_incv3-keras_features.npz     CIFAR10_vgg19-keras_features.npz\r\n",
      "CIFAR10_resnet50-keras_features.npz\r\n"
     ]
    }
   ],
   "source": [
    "!ls features/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load CNN codes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model_names = [\n",
    "    'vgg16-keras', \n",
    "    'vgg19-keras', \n",
    "    'resnet50-keras',\n",
    "    'incv3-keras',   \n",
    "    'Inception_v3'\n",
    "]\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "data = dict()\n",
    "for model_name in model_names:\n",
    "    data[model_name] = np.load('features/CIFAR10_{model}_features.npz'.format(model=model_name)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# It is important that CNN codes for all the models are given in the same order,\n",
    "# i.e. they refer to the same samples from the dataset (both training and testing)\n",
    "\n",
    "y_training = data[ model_names[0] ]['labels_training'] # this should be common for all the models\n",
    "y_testing  = data[ model_names[0] ]['labels_testing']  # this should be common for all the models\n",
    "\n",
    "for i in range(1,len(model_names)):\n",
    "    assert( (data[model_names[i]]['labels_training'] == y_training).all() )\n",
    "    assert( (data[model_names[i]]['labels_testing'] == y_testing).all() )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LinearSVC classifier from scikit-learn\n",
    "\n",
    "We used the linear classifier from the [scikit-learn](http://scikit-learn.org) library.<br/>\n",
    "More precisely, we used [`LinearSVC`](http://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# First we tried all of the following parameters for each model\n",
    "model_params = {\n",
    "    'vgg16-keras':    [ {'C':0.0001}, {'C':0.001}, {'C':0.01,'max_iter':3000},\n",
    "                        {'C':0.1}, {'C':0.5}, {'C':1.0}, {'C':1.2}, {'C':1.5}, {'C':2.0}, {'C':10.0} ],\n",
    "    'vgg19-keras':    [ {'C':0.0001}, {'C':0.001}, {'C':0.01},\n",
    "                        {'C':0.1}, {'C':0.5}, {'C':1.0}, {'C':1.2}, {'C':1.5}, {'C':2.0}, {'C':10.0} ],\n",
    "    'resnet50-keras': [ {'C':0.0001}, {'C':0.001}, {'C':0.01},\n",
    "                        {'C':0.1}, {'C':0.5}, {'C':1.0}, {'C':1.2}, {'C':1.5}, {'C':2.0}, {'C':10.0} ],\n",
    "    'Inception_v3':   [ {'C':0.0001}, {'C':0.001}, {'C':0.01},\n",
    "                        {'C':0.1}, {'C':0.5}, {'C':1.0}, {'C':1.2}, {'C':1.5}, {'C':2.0}, {'C':10.0} ],\n",
    "    'incv3-keras':    [ {'C':0.0001}, {'C':0.001}, {'C':0.01},\n",
    "                        {'C':0.1}, {'C':0.5}, {'C':1.0}, {'C':1.2}, {'C':1.5}, {'C':2.0}, {'C':10.0} ],\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we start to train so many classifiers, let us write all the results\n",
    "we obtained after hours of computation.\n",
    "\n",
    "We tried to build LinearSVC classifier with many possible paramater C.\n",
    "Below we present the accuracy of all the considered models.\n",
    "\n",
    "                                           Model \n",
    "             -----------------------------------------------------------------------------\n",
    "       C     | vgg16-keras | vgg19-keras | resnet50-keras | incv3-keras | Inception_v3\n",
    "    ------------------------------------------------------------------------------------\n",
    "     0.0001  |     8515    |     8633    |     9043       |    7244     |    8860\n",
    "     0.001   |     8528    |     8654    |     9158       |    7577     |    9005\n",
    "     0.01    |     8521    |     8644    |     9130       |    7604     |    9061\n",
    "     0.1     |     8519    |     8615    |     9009       |    7461     |    8959\n",
    "     0.5     |     7992    |     8014    |     8858       |    7409     |    8834\n",
    "     1.0     |     8211    |     8225    |     8853       |    7369     |    8776\n",
    "     1.2     |     8156    |     8335    |     8871       |    7357     |    8772\n",
    "     1.5     |     8172    |     8022    |     8852       |    7318     |    8762\n",
    "     2.0     |     7609    |     8256    |     8870       |    7281     |    8736\n",
    "    10.0     |     7799    |     7580    |     8774       |    7042     |    8709"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# and we decided to choose the best parameters\n",
    "model_params = {\n",
    "    'vgg16-keras':    [ {'C':0.0001} ],\n",
    "    'vgg19-keras':    [ {'C':0.001}  ],\n",
    "    'resnet50-keras': [ {'C':0.001}   ],\n",
    "    'Inception_v3':   [ {'C':0.01}   ],\n",
    "    'incv3-keras':    [ {'C':0.001}  ]\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model =  resnet50-keras\n",
      "X_training size = (50000, 2048)\n",
      "features=  resnet50-keras, C=0.001000 => score= 9158\n",
      "model =  Inception_v3\n",
      "X_training size = (50000, 2048)\n",
      "features=    Inception_v3, C=0.010000 => score= 9061\n",
      "model =  vgg16-keras\n",
      "X_training size = (50000, 512)\n",
      "features=     vgg16-keras, C=0.000100 => score= 8515\n",
      "model =  vgg19-keras\n",
      "X_training size = (50000, 512)\n",
      "features=     vgg19-keras, C=0.001000 => score= 8654\n",
      "model =  incv3-keras\n",
      "X_training size = (50000, 2048)\n",
      "features=     incv3-keras, C=0.001000 => score= 7577\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import LinearSVC\n",
    "\n",
    "# C - chosen experimentally (see explanation below)\n",
    "results = dict()\n",
    "\n",
    "for model_name in model_params:\n",
    "    print('model = ', model_name)\n",
    "    X_training = data[model_name]['features_training']\n",
    "    X_testing = data[model_name]['features_testing']\n",
    "    print( 'X_training size = {}'.format(X_training.shape))\n",
    "#     print( 'X_testing size = {}'.format(X_testing.shape))\n",
    "#     print( 'y_training size = {}'.format(y_training.shape))\n",
    "#     print( 'y_testing size = {}'.format(y_testing.shape))\n",
    "    results[model_name] = []\n",
    "    for params in model_params[model_name]:\n",
    "        clf = LinearSVC(**params, verbose=0)\n",
    "        clf.fit( X_training, y_training )\n",
    "        y_pred = clf.predict( X_testing )\n",
    "        score = sum( y_pred == y_testing )\n",
    "        print('features={:>16}, C={:8f} => score={:5d}'.format(model_name,params['C'],score))\n",
    "        results[model_name].append({'pred': y_pred, 'score': score, 'clf': clf})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.externals import joblib\n",
    "\n",
    "for model_name in model_params:\n",
    "    joblib.dump(results[model_name][0]['clf'], \\\n",
    "                'classifiers/{score}-{name}.pkl'.format(score=results[model_name][0]['score'], name=model_name))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-rw-r--r-- 1 rno rno 164764 sie 30 09:02 classifiers/7577-incv3-keras.pkl\r\n",
      "-rw-r--r-- 1 rno rno  41884 sie 30 09:02 classifiers/8515-vgg16-keras.pkl\r\n",
      "-rw-r--r-- 1 rno rno  41884 sie 30 09:02 classifiers/8654-vgg19-keras.pkl\r\n",
      "-rw-r--r-- 1 rno rno 164764 sie 30 09:02 classifiers/9061-Inception_v3.pkl\r\n",
      "-rw-r--r-- 1 rno rno 164764 sie 30 09:02 classifiers/9158-resnet50-keras.pkl\r\n"
     ]
    }
   ],
   "source": [
    "!ls -l classifiers/*.pkl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best accuracy = 0.9158\n"
     ]
    }
   ],
   "source": [
    "best_model = 'resnet50-keras'\n",
    "X_training = data[best_model]['features_training']\n",
    "X_testing  = data[best_model]['features_testing']\n",
    "\n",
    "clf = results[best_model][0]['clf']\n",
    "print( 'Best accuracy = {}'.format( clf.score( X_testing, y_testing ) ) )\n",
    "y_predictions = clf.predict( X_testing )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we obtained **91.58%** accuracy on testing dataset using LinearSVC classifier on top of features extracted with ResNET50 convolutional neural network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Some misclassifications"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion matrix:\n",
      " [[935   3   6   4   6   0   1   6  33   6]\n",
      " [  6 944   0   1   0   0   4   3   9  33]\n",
      " [ 20   0 878  28  34   9  21   6   2   2]\n",
      " [  6   3  19 834  22  68  23  14   4   7]\n",
      " [  3   0  23  10 918   8  14  18   3   3]\n",
      " [  2   0  11  71  18 875   6  14   3   0]\n",
      " [  4   1  12  19   6   2 954   0   2   0]\n",
      " [  6   1   6   9  32  17   0 929   0   0]\n",
      " [ 23   2   4   1   1   1   0   1 959   8]\n",
      " [ 11  45   1   2   0   0   1   1   7 932]]\n",
      "['plane', 'auto', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']\n"
     ]
    }
   ],
   "source": [
    "import myutils\n",
    "from sklearn.metrics import confusion_matrix\n",
    "labels = myutils.load_CIFAR_classnames()\n",
    "conf_matrix = confusion_matrix( y_testing, y_predictions )\n",
    "\n",
    "print( 'Confusion matrix:\\n', conf_matrix )\n",
    "print( labels )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We have, e.g., 6 cats predicted to be plane\n"
     ]
    }
   ],
   "source": [
    "i,j = 3,0\n",
    "img_idx = [ k for k in range(10000) if y_testing[k]==i and y_predictions[k]==j ]  \n",
    "print( 'We have, e.g., {c} {iname}s predicted to be {jname}'.format(\\\n",
    "       c=conf_matrix[i,j], iname=labels[i], jname=labels[j]) )\n",
    "# print(img_idx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+8AAACPCAYAAACLZuriAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvVmQJdd93nlyu1vtS3dVr+gFQDcaO5oECBIkQYqyaFKW\nGJKHGvrBYUXYfvKbI2YLP0zMw3hiYsITth9sx4Rlh0IOL7IlzciyZFuyJIoURWMhSOxb71tVddde\nd8nVDy3k//v/b2WyGwQbd1jf7+ncPqfyZmWePOdk1/edzyuKwhFCCCGEEEIIIWR08T/uEyCEEEII\nIYQQQkg9fHknhBBCCCGEEEJGHL68E0IIIYQQQgghIw5f3gkhhBBCCCGEkBGHL++EEEIIIYQQQsiI\nw5d3QgghhBBCCCFkxOHLOyGEEEIIIYQQMuLw5Z0QQgghhBBCCBlx+PJOCCGEEEIIIYSMOOG9/LKj\nx88UVXWe55XlINCnFQbRrnW+76l2USTtwjBSdZ4XwM/5u5ZvH3/3drucsPxMXbuhH7uztng9vLzy\nsg0dD69J4aScuVy1S+JuWZ6fnVV1f/lrXy/Lf+Wv/qy+yHuIX/3tPykvfJZlH+ep/MgUTvqQ58wt\nrbvD1V3vzr+7+BAH8fRJ3Wkn/OWvPb9n++tzz32+vNCNhh7/xjtjZTkM9JiRpHFZvnDpYlm+dv26\naofDredV96ECy+bOdZqNshzrIcl94sD+stzobZTl9Vg/ew8+frYsL3bPq7p9n/nFsvzpL35Z1U2N\nyTX4v/7e/1mW/8N/+A3V7sB4pyyPw88459yJk4/I8Z//UlneNv8P/tbbb5Xlb33rD1Xd8Sgpy7/x\n4lt7tr9efPOFsr/e2lhXdf/1pZfK8unTZ1RdBOuB737nz8ry/NycatdsNsvypUuXVd3Otsx/U1Mz\nZXl9fUO129jYKsuz8/tUXQHz/mAwKMtJkqh2MzMzlXX4c3F3S9WNj0k/3BzIM7Cxta3a+Wm/LHea\n+rk/dPz+styVx9xtrt1U7Yp4pyxPTIyrutiX6/h//N3/fc/2V+ec++rf/ptlnx1rtlTdWLtdljut\ntqrDtWmjIdezadapAYyrDVPX78l9/sNvfrcsv/H+RdXOwc8Veayq/FT6mJelZTk3Y7EP51GkZu0D\n07ln5vY8kwMVcNDcfAGurW2H8p2cV1LId3fm9Tr1xP0PlOWw0O8MaVeeq2/91m/v2T77P/wv/2N5\ng8YndJ9chOu5leq+3B8/UZYbCzKGpH6g2vk+vi/py4z3XL3fhbpdEEgfCgN9fA8WE/hTvll/5J4c\nI3e6v3r57mXnnEuhv6bQle3xe5syL2TmeZiYmpZjwL9n5r1NfSrstZLa//mnj9xRf+Vf3gkhhBBC\nCCGEkBGHL++EEEIIIYQQQsiIc09l8ygdGgZl81o6gVIKVDMUhdZAxIlIZdJUy9MKkCnkBco5jLQh\nz3Zt55yWQeQgo64TBQdWJo9yJKtVAlBWFPj6G/CYvq9vIcro0SpgrQj79s2X5TMPPazqOkYquldB\nubeVfqOu5SNQln9o1HkYsU2mzkzKvhGqefYHkTv85T6a61HsWvzRjrl36HW78EnL4How7hyAZ985\n59pNkX0dWlgsy5evXlHtXnn1+2U5rbGR4Ng4MONwG+aAItZ1bywtl+VofFLOPdXj5N/6xi+X5Qu/\n8Q9V3SLIhKenJvR5wZjahbkiMrandiRj5c5An+P0mFzXNAF5p5mz1tZW5Ri9rqpLPoyN5CeQP/7m\nn5blm+trqu7musjo1zd7qi5LpH+tr0q7NNNzHEret7a0JL0FkmcvFGnxxlZftYtBS5mZ2xaD5B2P\nb+12/ViOr59RLZv38lTVoRRUDdFG0pmiPFmfouv15ffZ6ct1801/9fG5NP1zMNDXZC/TUPL3hqmT\nz6FZb0UgZR9rSd9rhLqdjzJhcx/mp8V+8fxnnynL3YF+Pi4urciHUPfFHOYFP4QxPDN9D9a6do2J\n43uW6J/zcK2O1yDX80WBfdj056yQc0a73+z+A6rd/gNHy/LWhraSdHurjmjpepbqe5XC59TUZdAf\nCrAuFLmRexf4TqS/G8cRlM3b8UW9Zll7Baws8Rh2nNMr3erFo33nws8eSOgjp6/HW6+KPctv63XF\nyUc/WZZxreKZ98fQw7lEP5co379T+Jd3QgghhBBCCCFkxOHLOyGEEEIIIYQQMuLw5Z0QQgghhBBC\nCBlx7qnnfXVVvDh1voTARBkp+xB6143/wvfFc9QyUR3o2xiARyjLtacRTyQK9PGbEL8Ugd8ptL4l\niExomliEZoRtbezC7hF2sYmXceBrD317C6EuFH/T9PSCavW5zz1flk+AT9Q55+LB3fsvfiLxdvfb\nDDW7F+fyIdD7OVRHD3pgVrLRg4WJBrmj773rn/jRf5I4d+WqRAbFsY4ICmCM8h55VNU9/tAZaCf3\n+9jTT6t22DX+7IUXVB36bucnp8ryFx59SrWbn5F4mk5D97U18Dn/3utvluWGp8103WsSD9fs6H1U\nIsiCyc34jfGY/e3NsjwVae9qpyHXYGdHf3c0IX439P8lqfYFr4GHe7urPalpxH7unHOXry6V5U2z\nLwBedfRqO+dcMpC+3U9kLBukNj4IfNy+vscJ+mphgdGe0H7GoisRaqnxMKIPvdmWuTYfigjCGCOD\nj+sZWyfnGEGfbLSaqlnmZH1gty9JcGyHdZWXV//dJjXrjdgukfYwLVj3NSPreZf+ZqOmWlDXhLHY\net4xOm4o0hU6yOEFiUX88k89p5r9+9//o7J8ZfmWqisi6Tse7KVUpAPVLgcve243e4B+WZjzV/Fg\n+L12byY4hvVbx+D/nV+Q+NCHH9Nzyew+WdNGbR01meQcY50b3j8MqV3TVkR3F15Q2c562XPleZd/\nt5FyaouQoXO6s/uol7qmT6q9Q+zeVeBzh2u1dlFH0HbPv1uWj5x+TNVFMD0NPBgTPD1wep7MW3b8\n/TDb4PAv74QQQgghhBBCyIjDl3dCCCGEEEIIIWTEuaey+dyJtNCqI1BG4xlZhYp5A2mZUdu4Jsgw\nOx0tLWu2RM5w66bIF5JEy0s70G4o8qUH548VJiMhTeVz2tSRTf0BxGyYuCWMvkMZRWb0dHitolD/\nnq1WR8oo0rNxHHDMbl/LOn3vnnYL8hFhnylUDWMkRsNI2NpNsZgM+lo+F2NcXo3Minz8PPH4E2V5\n3URvXb5ytSyHNoozks8hyOZt1MkXnv1UWcYoNOec2+nK2PiNz32xLP/UiTOq3dbKTTm+UfS96V0r\ny7/n3ijLgREbb8Ax+j3dX5O+yK9zkDw759yrL/6nsnzl0vtyTkbmPA+Sy4m5fapueUvGylf+4D+W\n5fNXLqt2a7dErhqZuKbDUx1HnIsxqarQnSFDOaavrRE+9teG3DuTKOiaHYk8TY0uMctE0uiDta3R\n1t+11ZVFRj/W1ogmzO0tkA/beR0tK9Mz06oOo+O623pBg5LXVkPm+djImJMcngGzFsngM/6YXxN9\nGhgp9ISxIO5lmqGsD63kvRFAjJyJRUbZfAvG2KFjQJ2z9jZl45P7evKIjlD7qedknP7dP/62qru5\nhfYUeWBS69nADmH/xIdNzVo9ANk/rp9TEymHMcyJ+e5mW/rb42clhuvg4ftUuwQ6dGdyUtVN2ZeD\nPUoU7B6755xzCViJTRq1C6Cv4X20K8ACbGpW+Y1Rh0GOx9Pt8LNvZO14XkM2EnUecI5m3eLDOVoH\nUABzS7Yhc/bKq99V7Y6BPW//9KyqyzGSG54Hb+iRitQnXXf3unn+5Z0QQgghhBBCCBlx+PJOCCGE\nEEIIIYSMOHx5J4QQQgghhBBCRpx7am7u99EzbjT/6OcxPhqnIgeCXcvOOZem4l/v9rTfEb1f6Ldp\nN7Vn/OQJ8dWMT0ypupde+n5Z3tjcLsvWG59B1EWzaX1sIZSrYxzQdxYG+jZF6J9qav9kEz5H4M+y\nppYErlWaaSdIYK8/+RiAeCGv2oCGFqHceH1C8MxNw14OB6Z1vw7gWVxe057J5QHEEA35dKo/aT5E\nDga5a04elbGr/cCDqi5+XHyxs8Z3G8LzPjkuPmHrx+zAWPnLv/R1VdeACMzFWfGEXbyxpNrdWLlQ\nlnfM7HMOPGf9TManotBj6OVN8fNPpnrsWrtxvSwvX9ZxL//5j14uy+9dE89+4XSff/XWVlkOtvR3\nv3X1j8ryNPgsDy0eVO3OPixxfH6mj3G0d84R5zY35Trb/TQK6JNxV8fIYTxqA/YTiGO9dwvGt9nE\nS7QTp+BRt3FHjZYcIwiqY7EwR9GOkxns2RMEOl4sDyCSy3j7B2rNIs9Ds6nbJX3w25uhNmqIf7iI\n0f+u94rw4ZoWZv+B/ra+/nuZANZ6gelUIdQ1TF8J0UMM87TvrMdXPttlmA99DOOUczO+PHFKon8T\n4zX/nT8UD/wG7Ami1orOqc2lcnMMjB/EaDvnTHQYPGSF8e9n8NnGOj96VvZueeC0zGOhWWfncB5h\nQ59HG+axvQxeldR63n2ISTOdDWfV1OH4aO6B2hPJfru3S2l4byaV8ubVvBcOf8GuFGaPHC+H/mvO\nHzdKufjyt8pycP111ezUk58oy5smqrMPe7B4TsZp3+x9kmTyjGVDa+LM3S38yzshhBBCCCGEEDLi\n8OWdEEIIIYQQQggZce5tVFyGURd3Lo8AtVB9XADIkfo9LaErQMreAIlQPNDy+vfeF0nj7KyOCUJJ\nHUrl81zLivBXy00MRrstkotWS8fIJbGIVZogUW0YmVwUyflb2XynM16WUaIX+lrqgXFwnrOy/7uX\ncBDBSi8/YFhVtLusyDkdd5GCPC8wz0YTJEJ+pOsmoZ+fnJ8oy1Ohvr+r6xtlebqp+8LNBKKYbLwQ\nSpC8uv8HrO5PVdfKYscLMgzGVU209dgyd1DihMLI2HAwuggkYJGRIuLnKNKS0XYDZJcwtoSLJmrt\n1o2yvLaxperGFhfL8jOP4f3WY+gkyNWzm/o8tiH28s9efEnVvXdBJPWnHhRp6flzF1W7AUSCDXb0\n/DA9NVOWp6bEfpAZ6er5y3LMfGdD1T15Qt+bvUqaoR1O1+WJ3PPt3qaus7FWf86QXNLD4+vxScVY\nDfqV7ZTc04yvDuK6MIYpMZl1BfSNNDbRhjDn26EQp+EoqpblYyqW7+m+lYP+NYB5PjO/Zxbs/vw6\np6Np9zo3rskYsm9uTtW153C81DczxDkc7l9gI7oqfsY55wK4Z2hT8p2V50qHeObRh1Xd9rb0vz/4\n9nfKcj/R/RIfJd+8JkQNsGkk2raEa3C0j+ap/kXRMvrAiQdU3VNPPlmWO7Be9s0yAl2Cno0Yi6ot\nqXsJtPoUgRmXUPJuBh8V5YdxcEZ27mfy2TfjUgERnINQjm8l+iHYn3On7xvK8n3olEFhYrbhuxNj\n+8GBNMp1NPj118VK98o3f68sP7GorRzz+8RqutTX83k8EAt1BPbUzPyeKYy/Xv6jv2NxVCaEEEII\nIYQQQkYcvrwTQgghhBBCCCEjzj2VzX/YXadR1q3ls1bqgVIEI6sA6UQQirRsdnZRtYtjkVWsrWrp\nfRSK9Lg5PQ41Wo6Swg7IUaTl6lOT07u2c845H+Qj4+Ny/GZDSzgaDTlm08jmx8bkHPV101IS/Izy\npg9+knz02N5fvz+73B8fZKLzWsnsjnSkXZFpiW8jkn4+nsuOwV6s+93+aelDXl9VuSbIkNPY9BMl\nVeOO8h83k5Py7M+YRIEOWHTaRlIfNcFeAzI7lEc651wI8nq78y8mYAQgr5+JZlS7g/sWyvJrb7+l\n6q6v3izLDz0kuwzvn51X7abg93ztlj7Gg488Vpb/n3/xr1Td2qbI3R44dqIs35xcVe2KLennYx29\na/EgFknqzTXZHX+np3fkRgnhXEM/6e1Ij+d7lfmDIpe1thickwojk0dbF86hnpFEZigFNrJQlLmj\nDD9Ntf0Bx7WeSWVxkIJQ4M7w5hgD/O7CSldBWmyG0O2BPKch7Opt5fUefF+jaXbEB0sA7uhvd/dH\neW0YmR3Eo3u8TBxhXnv11bLcivQO7U9/8umyfBySP5zTthBM9wgDPY7q3eyNvFjVgWzeTMs+2Dk8\nMy8/Dru3X7xytSy/9qYeR3H8srvq487xcWK17GAfifF51Odx8MCRsvzJp55SdZMTsvZNIHVkSL4P\n1y4ypxGG/Lukczq5y6Zl4AI0HuixMwE7RCOXhnmu+yRanH0jZY9hvm02pW56YkK1a/gyxw7M4Jag\nvUdVVY/TuU0MgbFz+8L7qu7iC39UllsprJ9Dncjz6nsXyvLShH4e5o/I+1iRVj97AfZJ886Vp7tb\nwepgDyeEEEIIIYQQQkYcvrwTQgghhBBCCCEjDl/eCSGEEEIIIYSQEWckzUwfNhYK4w6SRHs40Jae\ng2fHN7Eo7TZ4zY2/GH8Ooy5abe1rj+MelLWJuAGRStZrPjUtEUhHjx6Bdtprh374ORNnh14otN5t\nrGs/NMJouLvnTiPOkPp+PRQWV5ZC8BLNt0wcXF98wt3VK6ouHJcom15ffLabxp/rYA8Ir6U9yp1Q\n+nZ3YOIXwU/lwfWwv0ndlWIE3EfHJO6TEWhfVqsl97Hd0ftkqFhKGJ+CUB8DfYR+zX1Dr6aN8GmO\nSV87+4mzqq7blX7Zact5tNvad76+JhFzcar//7kNHvWLV/Tz0O/LWLwOEXCHDh5W7cJl+W4bk7XT\nWy7LJ/ZJ/F6zfUS1W9sQv99srsdeG7m4V0kK6QtRqP3DYYR9qDpWFkebwPR5jIOz/RBHJT8AL7iJ\n8EFPvd2jJgY/eZbKemNgYmpTWAMUZi4vIGY2z/SaJc/l5/oQaZubc8R4u15frzd2urKfQ6HWLzYu\nMoQ6sz+OzzH6Ax44eaosv/jCC6ru9TfENz4zp9dl7bbMvx3w4PrO7C+Ae92Y++Dg8+ZA+s2Na9dU\nszfeeL0sX71+XdVdX5b1wsraWlmOTb/B/RcaDb1HCg5fyUD35wjaZtCfJyf1MZ7+1KNlefGAXnOo\np9uX65OavS88X06kYfaS8LtmjbNH0bGa1XsXZOb6qb0XchwfTcQkfIzMn4JjmH9vnZc+GU9rz3tz\nVvbBmT7yoKprTcpzlMA5FZlZmxTSD/NUj6P9LRmPr775fX3+a/LsBLnskfLOJf3cXIJU29N/4VlV\nN9YRf3xvR56pwK58ob+6QI+pxYfYo4F/eSeEEEIIIYQQQkYcvrwTQgghhBBCCCEjzj2VzaPcrU4K\nVy+Tq/73+mNIOYeIlyQd6HYDlNrp78O4ljAUGVTga+lToyGytnigJZNbm+tSZ+IBcvjCpZsSQxQa\nyd/OtshAtje1RK/ZlPNKEowt0OeIkkIrm6eU+TZ+TbRO3TWqqquX2pufUX1Z/rkZ6kc23RB5WLKz\nqeryUOTRL7zyg7LcBpmSc8498YTIgNptLWHb3Jb+u27loEo2L//um1+zgN+NguEfHyoOrqmtPO2W\n9IWmkUGG4e7TgI2DU5J6E3GE3Rf7v407QhlfaCxL0xNiG8pAGpz19Rj9B7/2j8vy/Y88qeq+9V9f\nLMs3IXrOOecGAznO+YsSGXP8yDHVbgrOY+XmiqprBXINvvG1X5DzOHFCtev1QIL34u+ruuW3v+OI\nc32Qocd5tXXLjqc+yMRxHvPNPIyd0nRltQbY3FzDGtUuyzGSSPfX+XkZR5uwNmhPmQhb7MtmrsXH\nwytMFF0q422u5Pb6+HiOea4jlFDqn0F0V5HodlmK8n1TR1tdyakHRDa/dH1J1V29JlLbF196WdWd\nffyJstw8JBYbG8fpQKK8Y9Z2Fy9dKsvf/NM/K8vnL1xU7TDmKzf3LoXPOUQpZkZqnEEEWNLXdVOz\nYsd7+OHHVN32jsjvbyxfLstHjs6pdoeOSERz4el+H8BaNYBI43TIWSjPwfI1LXN+8UW4/n/H7Vly\niPkLzd9qvQGMS+bnPPg5jDzLAxN1CWNsahZ3+w6IHS3qyhjrreq4tuKGxC+ef/mbqq55+pNlufOg\n9LWpcS29z66dK8tLr72k6q5duFCWk3U9nzdiWT/7LXkWN2Ld59e78iz2t5dV3cq735PjNcS2N2as\nMzjY++ZeNCI7d/1w+Jd3QgghhBBCCCFkxOHLOyGEEEIIIYQQMuLw5Z0QQgghhBBCCBlxRsbz/lH8\nHHqKMT7FOe0SU5FyqfF3QayPZ3zIGFnUhGQbG93ig7dhZmZa1V27LtEEhTnHZEu8Pzs7kk0w1tL+\n1clJ8WRev35V1S0uHCrLJ0+eLssT4/o8IvBY9IyXOYp0bM9eBWOifONXr+2HFR/M7XaFw3g1Ex0B\nx8/Rf2a+qzMzX5bjZFvVDSLxNk8cOFaWpxaPq3Z5AyLlYj0kBE76xpB/GdI6PPDq+YX5XSrK5KNl\ndkb2K2g1ta89aoAn10TFhZHc8yjCyCjdYaNm9bjg+7vv0VAY/2wIcUeFif7JwTQXQe5M1JpU7X72\nG3+tLG8Xur/+5q/8alm2caHYfbe3ZX+InolORB/fxIT21g0Gch0XD0pUXBDp6Jq2J9e4k+kYpssD\nEwG1R0Eft1dU/x3BxvXhjcSpt7uu9zhYhrkWPe7OOZckck9WVm6U5f5A9wUfnoH9C/epusmW7C/j\nt+V+50P7l+A+Orq/+io2bFzXRTK2Y3STnUcwMmvY2y/PWI5Rd4VesxTw2UbipcYPvZeZn5sty899\n5tOq7k/+RPy6F957T9XdvC597AB4gY8e0RGT2+sSMXn+Xe0N3liT/ZL6sFdTZsZRxM63uOeIqvNM\nPCAeYyhlUf5haUl7iAPYZydPpN2E8Sjjmtn2rzDA/aQCKOvTwL0Y7Bp2Y33dEeciWAd4nnnmYU+s\n1MRWR2ogwT5jjgGvkIl5nYybMm9PnZK9acKrenw8mcq4d/Wdd1Xd7//WPyrL/Tl5tzl63wOq3WBF\n3oPGQt2fOrC3w83tDVUXp+DtD+TZjswQjntLfeff/UtVtwNrkPtOiS//7Je+rM/jvqNlOTeRe3l6\n9/uK8C/vhBBCCCGEEELIiMOXd0IIIYQQQgghZMS5t7J5lH592GPcody+MIIhjD5IUSqfGWklyPei\nQG/fX+Qgt4c4Ds9EucWxSO97fS3DyzI5/0ZDH7/dFrkQSjk7pt3UhLRbXtIRGfP7JJLk2PGTUmGU\nVSif8o3WLjVWgr0KdJkh+RnK6G2PTDHKCLTlVhiT+dKHGuYG5Zn83BzIlWcb+kzahbQbjGl5cZLI\n4/3wg4+X5b6JVNrakBiP0LVVXR/OsQj0bxpiFJOT88iNvN5z1deKfHQ0wO7SNFabhpLGa4lksy3S\nOpR/23uF40Rkok20tBnk74HuTzrySoPja9SU47934Zxq96u/9i/K8vKKlko76NvzMzqe6MD+/WX5\nDZDnbW1tqXZz8xLxMj6hpcyDnoztM9NiUwjN9RhAvFI+0BGO/Ug/p3uVvEBbklmKgNzb2iu2e3K/\nlq5LHNXKsp4Lt9ZlXIsH1hohozHaRgJjGWuPyz2emdP9CSPrcpCdDkeCoqXERHeBZNiz0mX/Tu0V\nML6auRzXJlp6b8ZoiEAMQj122OjPvUwUyjU8eVzbKGYnv1KWv/PtP1V1V5dknLp0TST0S7e0vLu/\nI/LvpKsjMnEt4cE4V+R6DavWyGYQR5tGlZX09j9AvKuxtExOyTPx6ec+q+oOH5Q1541libDrJ3qc\nRpm7XdM3IuizEGMa2F8GztE6B7K02kqwl8BLi+8szjlXeHKNPLO2UzHJGdqRTT/BIcrcxwHGUzen\nymJ4+CnV7srSO2W5fVQf4wDEvF26KjaSc1cuq3YhxN9+9S/9JVVXwLPyu/9Rx7aOzYhU3g9k3Fu+\noWMgd9CWYYblDvTDm69KTN23lm6odoeelN974ZGz+hjTs+5u4V/eCSGEEEIIIYSQEYcv74QQQggh\nhBBCyIhzT2Xzfo3kvW5HeZRr1rVD6Y9VAVXJ7XOjt8HdZa2UaBDjDp/w/x6elobGIDvv9rT0CXdz\n7g/0LsT9vrQNA2m3uqalVQns4Hzs2AlVNzcnO9Si9D40UmmUvQ7L/KiTc07voG0vEe6oPrQTfQBy\n+FzKHSPJ9EEq38nNrtxtOeZ0JHWtrVuqXQyyy8mWPv7m9mpZvvE+SEiNRKrZkh1i251FVbcTyjHz\nQOuFQpDTqZ1g657zyhryozIBMu52u2VqQSLn2/EV+jnsehoNSe9hZ+xQ/7+vst7A42B3rMfxNUt0\nn2+05fgD2I34H/zjf6La3VoWCebyqn4ecIx+/KHHVN3hgwflGOsiZUdZ4O1/kGu1tb2jqk6DVHYa\nZPmDvt7tOAd7S2bqJg/qtIe9igdznJXmog62ZyTvly9dKMvXrkk5Huh7lcAuw5nZ0RqfgAQktu1w\nTLXDHebn9x1SdRn87QOfGzuf4kdbl2Eag3kucRz1YextNMw8gs+eSXfAY+SQCGInNAwIGZIgU4Fc\n0gAZdzPUy+f7DsIu8r/wi6ru4hWxdLzytuxEv27GlxUnc/ZgoO9RDvc2BbunTS1S8++QhQ3Geh+s\nHl61RcMP9O85CVLjn/v5r6m6+RlJ4Ph/f/vX5dw3tDWp1YRd6Y2fEO0jnsM1hkmQgjGjZywGQ0va\nPUq3K2On55sd5cEiZ9+DcJwKfLQu6H7iqzglfQxcZ3gh/FxzQbcbE2va+R+sqrqrufSTxJfELM83\nqUgdGbdbHW1vOv3II2X5nQvXVN3lS1fKcgzPYmT62r4DYJExPqJiIPP7OthgVq7pJDB/n9j2WodP\nqrrWpE4DuxP4l3dCCCGEEEIIIWTE4cs7IYQQQgghhBAy4vDlnRBCCCGEEEIIGXHubVRcjV/9Tuuq\n/O/D7bQ3A+PP0M8x5JuHuKK4sBEccowAvMy50xEMORx//6L2EE9Mij/+nXfeUnVNiDu479ixstzb\n1p6/MfB3LC4c1ecItpMEPfUmYgxtUoH9PxyPJjfnnFPWcOsdg0vUMFbFyUKu+1gmXqxD8/tVOy+V\ne3LzwiVV1xkX/2+nLfcn3dH7HxSR9JmtLb2Hwq1rErG1tgmRRB3tZQ5nZJ+EiWl9jqvgaSqGPJng\nTcPoQc9k5HDsAAAgAElEQVT4Lgv0sFnXO81pHxU+eDDtVQ3AtxiaqDgc80Lwptk4uNq9DNBmCb52\n3/gx0c8YGi+lBz93/i2JhcliPQ5/9dOfKMv/8g/+RNVtdcW35hlv2sS4eOvGxsRLt2b2FNncFn9m\n0NDPyn33ifdNxZGaZyNO5Fnc2dae96NPnnbEOR/6gh1asE9uQJSlc87dvCkxPmkKvvbCRiGJmbbR\n0n058ORzozlRlvejt9E5Nzd/RD54ui9kcI559aOhxryBiWtaWxOPZ2zqmhAdOzYGXnzjT45C3JtF\n93nczyKviexEf+pQLKrPv/F8QATjYzM0kZtwnZpmX4KnHjpTlh9/TCKj3jdxUv/6t/6/sry+vq3q\nQoh582Dpnpsb5sGYO9wt5ZnwAojmNGN7inOCGQOvXZdz/te//m9V3fPPPV+WX/n+y2X5wVNHVLv5\nefHGb27o3zPw4fugr9v45xzM8kmijfO+f09fbUaWQU/moabZE8nBvjWFuf84nnmw34Z9V/CwPxV2\n3QftIAo0M4NlCvdq/sSj+hRnXyvL/aU3y/L9R7VvfpDKvN9P9Hxx8LB44D/97CdV3e+sQNtQ4uwa\n03rPoBTW6rGJF1/vw94RE7Lv0Jmzn1DtHvz85+X449qXn6d3vw7mqEwIIYQQQgghhIw4fHknhBBC\nCCGEEEJGnHuqLQlA1jkk26qQxtvP2M5KxObnRf7b6ejIl3PnLsjxQFY0HN0CMhDfynTSXcuBid6K\nY5G/ra1rKVynI3EiZ04/oeo8D+T2CxLHcePKkmo3DZFQoZG2omQf4x/q4uBsXVWs3l5D31cbuSJM\nhPoeLG6JDLe1tVGWp2Z0pGDfieRzc+26qht0RTo2e7/EAfZMAhhGTHTXN1RdI5BnYAL6iZXZhanI\nolqB/oJpiDbcTnVfRu9FgJJM89yoHKLCSu/Z1z4qMrAG+U0tkUM5vG80yiHI4nA8SUzsVKcjUnPP\nynNxXMYOZhKI1BNlopZQuvtvf1Pko+ev6HiX47PS5ychZsY559bXRQZ38bK2omQQ5xWDFL8DEnrn\nnGuBXHlza1PVTU6KxBrHB8/MWWlXpPcDI+l88Cij4pzTcU55oa1aA4jfWV29qep2ujLOFQVGZtnI\nTunLrZYee8c7k2V5YnJfWV48eEy1a7akf1m5Z6bOuXocU3JfYyOZgtitIbk6PLMomXamr6mkQ3P8\nXElXIS7SOOPUusHGovrVMWJ7jUYk19fGSUUwfllJ/RiMdUFT5vblpl47FCBDdpG1M6IcXo7nD2X5\n1VhQ0TKKg7OdlqE3+qGeS1LocP/+d35b1b0HVtAFWMM2jPS+35W5ZcJIiKNI2mZwXlmhx1Fctw4G\nOiouYJ91zjmXx3LNWuP6nSgC21pmlnYpSOBzsB95Qza4GhszNEUrnbUYFTBOdzf1Pd7ZFNl/EEo/\nH3T1nJCBhSIwr7Xf/mOx1tnY7a/+7JfL8uSUWEZXV3W04QvffaEsv/v+eVW3HUh/ffILP1OW7//U\ns6rdDkajmkE2CO7eqsy/vBNCCCGEEEIIISMOX94JIYQQQgghhJARhy/vhBBCCCGEEELIiHNPPe8R\net6NPwJ97dbzjm0D8GeiP9055/bv379r2Tnnrly5XJYTFcli4+Yg1sr6x8Drgx6OINC+pYUF8dBN\nTEypumef/pSc4+w+Vffi914qy5cvi89zrK19yBNT4rv0A2sqld/HXh8E/ULDUTD0ITvnXIA+XtNf\n4fa7yNyDTiAesawvXqyNWyuq3WYqvs6by7dUXdgUj+ax+9EDpu9pkmFEm/Ehg8+sMw6+NeOv8cCD\nt7qqYzZC+LmpMe2Z2snB7+Sj/1efo2d97uTHQqsl40Ro/OQ5+AWLTPeTMJT7H8IeB5GJO8Kxdyjw\nDz1nQU3cHHprzUEy2KNjZ1O8aZtb2n/2Fnjg56Z0n9zYlLHR+tW/c11+Ds/rsUceV+2eeEL2Ivmt\n3/wNVXcCIjyjBkQ3mdi7HCIdxzvabz1jxv29Cu7J4js9JuH8lDs9nqQpxF5CLKXv674WhvI8jI1P\nqLqxjnzGOX9nZ0e1K8DPaCOzInhWsM8HZj7APSbs1jNqvxnrBa1oZ6f1DAzstVGccL29ofOAD0Nr\nM47fH4D7JdlVU+ijH173gRAucAeu53jD9llY69pIT/yQwbcX1dGfhekPGCfqw/xd5Hp/kxw6yNA6\nuOqcnHOXr1yQdrk8SwuL2td+5LDs+1FkeuxsNWVMT5ycV2aM2QXEoMUDXcd4w9tsb0sM38yMnnda\nTXiv6Oq4vg7c/wD2vknN/gc+zHuB2ecB16M4vtt3vxY8DxfOv6HquisXy3ID5oHlbf1dn/rM82X5\n4BEdxXr9+tWy3Gzp9cL8fumXTVgHT0/r97b3Lsp5jJsB+OxzEgE3flAiEbuF2dMnxz2iTKzeh+iu\n7OGEEEIIIYQQQsiIw5d3QgghhBBCCCFkxLmnsvkmys6MEgtlLsOys+o65PhxidSamZlWdSiT1LJ5\nTQYSkcLKkVQEB0ikjDT0xHGRBD326MOqbgpiiV773guqbm5S6r74U18vy9ev31DtNjdF4pKm1RED\nGJ+AEsWhdkYGUneN9xKBkjsaewVI03omQm0dYokaEOUWG2nS1qpEHm3d0DFv4wdE0hSA7Dbb0u12\nIJbOC/V5BCixb4vcaW1HH8PtgFQ61hEcY06+e8xEjahwFh/tJkZMV6u6LHYpDYPS+6HDUdXpnNPj\nkGekrhFETaE87PbPwTQAP+Zb+Sx8tNFeXoC2J4hEqZHxWrvJP/2Vf1aWnzp7tix3JrWEbX1VLCZd\nI6k/eURka71NXXerKzL6MJJ+bW0E/+bf/HpZ/unnP6/qHj1zpixvbch5NDs6sm6wJvGek/MHVN34\npH6O9ioJjJuFmYNSmKMbkZZqNhsyNiYQwxf4ul83IpHGh6G+P2FDPrfHoF1LH0Mrhk10rIrdkuch\nN9FdflFtCVQxuL5d9wS7thu2+sEZDkXfQlysktfrdnVS64y2pxKc2obHNr+i7FwWyucB2MpmJydV\nu1/48lfK8hsXdNTlf/nWt8tyCpLcIrX+I+iX1qeBnQWiLjsmShHniJ1uT9WlGUqgY1U3GMjnc+dE\navzIGb0Ofuy0SJvff/d9VdfwZQ2O8Yap6bN4he36dtg+sjdpo2XLjC8BzNPtQPfXZiL3PN2QuWx8\n7qBq50EccWr6PI6JuF4Ozb269OY7Zfm1F7+n6ga93d/Vzjz4kPr81JOfLsttE/16COx/SdxVddi3\n00TOa+mWjqeNwZL4zF/8qqprHjhUlrchmq9h4uAiFaFrXr29u38V51/eCSGEEEIIIYSQEYcv74QQ\nQgghhBBCyIhzj2XzIj2w8rEAd5s3ckrP332X+jGz+/WJ4w+U5YkJLZObm5kvy/2u7D4YGCkJSsiP\nHDmq6lZuikxyADtd5qmWDnWacr5zc3qX2xxkG5965lFVd+qUSIuuLcuu35diLTNpgOTTFdUWgDr5\nO/6eVnJkd6req+DOsoFnbAcgM0qMMm05F4nyvrnFsjzdtXJ16TfHDuq+tgm3LobvToxEbmtJjrl4\nfEbVdUDKuwZynrVNsxu8JwL4ucPaAtKBnbGLUEtKm6kcJ1GnZeRTKLGu2VFZ1Q2p/SiD+2FEIC8O\nQyODa8l9rZfuynX2PTMOo+LSt3V4f2DHZDOWYDvfSPVOPSTj3//99/9BWR7EenwNQHK2tql3lM8S\n6ctf//mfV3V92K34tR+8XpZ/7is/o9q9857Yr/7GX/+bqq7RlrF3piGJJt1uX7XbgPSIYFwnn/hW\nMrdHCWE3bavMzhNYD3g1cnKQG0ZGXt+ClJbWmF4PNGG3+RB2t46aWnIZNTCJwaQvRFWydk3Vs+Gc\nlrkXxopSZNIWp+i6NB57fHxWPNjJ3H4X2gDt7uLOft7D4JogtOtUTKcxP6fWW4nIwg/N6F3YP/3J\nZ8vyeqZ3pB5rSd/83d/7T2U5GtNzNh5/2sjyDy6Khae3LbvBP/2JT6p2x8H6+frrr6u61+Bzw1hG\nDx5agO+WZ+6R0w+qdvGOWAjHW2aOgO6G/T4MjF0EZNnFkGyeOOdcuyN9xr4PNCMZ21rGZhevy/zV\nWxdr5kSo147+lKxvg7ZefyYFriWknJrUgAS++7EvfUXVrV2TNcH5114uy9vmDq9vyzkmZif3LJP1\nQ6et7SEFWEC2NuTdrGcW9fMPSv+dApm8c86lME7PgT0xT3VySbcr73Sdab3zf1bc/RjLUZkQQggh\nhBBCCBlx+PJOCCGEEEIIIYSMOHx5J4QQQgghhBBCRpx7ar5rNcVv4BuPBfq4oob2lqFHMAWf7czM\nrGp3ALwInbb25+6bF4/B9WvXy3JuotZOPXiqLP/lX/q6qvuVfy5RRte2r5TleKD9jt0u+nm0x2J2\nQTxOB/ZrL2R3IL/bK6+9JsePTewTeO/qfO3aG2eiYTDmxviJs6w6Vm4vEWL8WV2OmQX2aNjEdIhx\n7aecPSL9dXpM741w5ar0r80l6a9d4425viV+nmRFR9F5Y9D3wD8X9fVjf2FZogiPfOJpVTc1L9Eg\naU/HbHiRHHPLybXqOu2DC2FfBhtdo3pezTXO/DtqtqdpNOS6h6GJhQnQ02qjpjCKU/qG3XvEB8/s\n8DF2/+DV+HNz41v7i18Rv9tP/8xPl+VuT4USuu0diXfZAD+ec9rrf/zoYVX3d/7X/60snzh2rCz/\n0n/3i6odxutYL3uK3koP5ilPn+P3X/t+WR60dNTdE/wvc+ecc0ks99Efyo6Vp9w3+zeg6zIp5JP1\nzs7Mypg6MaG9v82W+HGDUMZlz8TNuUKeKZNm5wrw3GI3t550fBbtugefPd+3c3n1M4vgnjU2Kg7r\n6rYNySFWtiiMr5V7NAjoa6+ZiOw+QrhHSJTKD06Y/jYdyj4NMxP6uv9Pf/1vlOUTi7J2mJnV6+Ck\nL2NRbjrtBOz9gEffN6f9yhgV98ChRVX3c1/6ghzf/J4ZfM7A2+w7s58D+PInzBq5n0pdALN9YMcI\niN6yvyf3yLlNoPZo0ONSAeNUEeg1m4MYz5km7Dlz613VbGNV1o7T951RdWFH+iV0ebXXiXPOHT8t\n71x5oN/98rNPleVTn5G16fU331btzm3KOR7p6OM3QulfntlcJYX9dOJY+t2ceTfzYX81P9fRiU2I\nNc36skaOd/TapOhKdG0708fQm77oOL4quIwghBBCCCGEEEJGHL68E0IIIYQQQgghI869jYqDqIsh\nlRzIO1Cy45yWb4YQVXDggJYXTE9Pl2UbhbIAERnRmyJJb5r4ly987vmyfOr+U6puEqTNSyA5GcRa\nMrm8siTfZSQiBw/KOdvvfvm1V8ryJkhD200tqa6TvKNkDy0GNv0Nj5EZyZHvG33gHgWjSby68BEr\nnyvknvd9kSPdNLF+vpN+vjAzrer2gfVie13kNgsPPKLabWyJ1OflF7+t6s48ITEb+4/fX5Zn9h3T\n5/uDN8ti98aKqtoBmb7fX1N1023pVGMTEhGzaqKddlz1dfRQGmrzohA/ra6jjt4551wQQSyUkeei\n3NPGt6ljBNXxVzrmzURSgSy0cHXjU/U9HoDkDKNlOg19vmMNkUAvzmlJeo7jfq47xqAnY+p4R2Rw\nGEXqnHPdrrTLUt3vUFJcgJejYa7p2Wcg8mlT21miIXn03gTnoMTKb2EKCptaVtsZl3veHci9urWm\nx6duTySR7faqqpudFSnw4oH7yvIYxMs555wP47d9pgoPJenV/RrnYYuWzdfFt8GzVxf1OGRnwZ+r\nPke8F7l1zTEqrsRGkqm6Gh29D9YJdQRzSyKwKJikODfVFsn72TMyt8dm/Tk9Ic/HoK9tPzn0xSYM\nQ63IjvXQH8zv5cOaNsv1L5DkUpdCf0tifR4JjKu+vabKwgF1NkoR2uWZXVtxUeCcfuatw9ZDO0+o\nZfMFdMyxMRkTJzq63eCWyOa33tPRaLPHREbfK2QMj400fnxc1r7plj5G5GR8XwQ5/H3PPKHaeaDL\njzL9PISe9DUv07GzOTxvU215z+oNdATtHDxjnaaWvCepfIb0Pdc0EY5RIHHluHZwrn6OqIKjMiGE\nEEIIIYQQMuLw5Z0QQgghhBBCCBlx+PJOCCGEEEIIIYSMOPfU894Gn6FF+7itpwui0cDDcfjQMdWu\n0xYv5Pa29hmeOfN4Wb586XxZvu+I9s1/9rlnyjLGpzjn3Bh4jtBnlhkfRQpmpUGsjUvXroun+Nby\nTV13VeoakewPYOPgdNxWtbcHI2rqPHl340vdS6jLYC9JzfVU/kHwt6ZOeze3PXn8wlx7woJJ8a35\nfenLm0va14kRdv1VHU1x5SXZ22HqwJGyPH5AR2g94T9alrvLy6quA96fsab+ndtgmstC8Zy1Gtqb\ndhX8SH0TzYjXNYfhyF5dHRNj+jy76xBBUO2LHfKyO/TFgTdzKNYPYnuMT1zFw+3+z0O1Q/e4ctwx\n5wHNklh7HfGgoYm4+vxnPl2WMVrIXo88w0i86rE3B/97YPY2ee7rf60sx4meR/oD7ininN7Lxc75\nKdyfqKHHzQg88JMz4iPc3tS+9gz6RmH201hdlbZeIEbFg2YfmiZ8V+DpOk95zWuerxq/OjLsmS6g\nTv7VxnPVRcXpdZX8e71v3vR5xm7tSp3HPTDXt4DxeBv83uO5WTvCpX/xtbdU3bWVW2W5v7NRlptm\nv42iIf15IjK+25b4l/0cxk5zHipC2dz+Ata3NuIxhZ/zIXK4aff5gD41MOtsD7ztBXjZh3qhOv/q\nY+xl8iKFsr5GaSr3vNHUfUiNARBfaONjp2FI7Me3VF28JLFyRVui1/zJedUuS+W7x1PtV5/1ZR+c\ndiLe8txEJmM8s0kJdy343VqhnksSiE1OerCHwkDHIk/CPBB3tR/+xtXLZfmBB2VvqWbbRJ7jGGvi\nlONBzZ5OFfAv74QQQgghhBBCyIjDl3dCCCGEEEIIIWTEuaey+bAmoqhONo+qyWZLNBFHj9ynmgUg\nr09SLWkamxDJ+zf++2+U5fU1LRN+/a03yvK+/VpS3wIZBMpLE6Pc3IK4g3/+q7+m6noQAXfmIR37\ntX9B5MyBJ7IKm6RRGw2DUlG4jlZqh9fbyvLT9O4lHD+JoIrXpph5rtq6UPgQW+EgQqswMRsgJbvZ\n0F8wNruvLO/rix1ka21LtWsMRPr0mbOPqbpVkNl1r0ikRzvVcp4AwmvG57WsyEEUTNoYV1XJmJxX\nsy0/N21tHqEcY9tIhgfQLWO4jKmRJGIizZC4lKkwzjkti7XjAj7j9nrhcIvjBMYb3W4I8ZJGpohR\ncfhddZLbosbygz9n5wMdWWeOCXLJNNNj3le/+hU4iBTTxEZ0yXlZlXMOHRGjw7LMRp1BDJo5fm4z\noPYoMVjKAhMfVMDfFRpNbbebBenv5PRsWc5NHFUWy9iYma6G06GHMYdGbpuCJc7aSFy2+3rGzqd1\nlhVcEw3b13aPbayTxluwTpdtf5XPuamzEXZ7mju0y9kxq4CxwmvKOmBsVkfEvnf1Sln+e//kH6m6\npTWxenztC8+X5cdO6UjjDlh42g1j9YAyjkPF0BiLUcLm/qOs3XS9AD5jf06L6jHWefb4Mrd4NVFx\naH3yzFxC5+dt8LoUuZ53EpDN2zErBLtFswHzub2wQQRFXdffFmtHBLen3dbrYG8gfa9tIoHnpqRu\nfkpsVpcuX1PtllbEMnrq9P2qDh1ttp/EA3l5C0OZZwaptqBuQ+RiaKJrBwlcY7BWZbluh2uHwFoB\nP8Qiln95J4QQQgghhBBCRhy+vBNCCCGEEEIIISMOX94JIYQQQgghhJAR55563n1lUKzzbWnvBPow\np6clQmv/woJqF0HMy8zsrKrLPfFSJNsSt3Xi+AP6HCPx0yXG+43xSw3wElmv2gB8yO+de1/VHVgU\nH31nYlLVofk0BC9J7oZM75XfXemTy62PrdpDZ/3xexVPxVrduYlK2bnMETWwb0Kk8y12GnJ/xsAX\nGWzrKI2Dc3PyYb6j6hbvO16Wwxb4ecwmDWkoJxybEaEBey90TZzF5mXZL6ID+RyTU9obP9GUZ2rM\nmJQT8Fql8H+JSa6vVVJIu9h43wYFo7ec+yH+8po9LnzlL68+xlA8HB5D1ckxrJULfe6eibOrCgO0\nbjAVRWd+FzQ321EsQ/8f+DiV/9LZiFDrpYSxF8bQNNHPZQYRR3Y8ZfLWbXBeKwo9Jqk5yVpi1fMP\nfvJQjzvowayPRsN25tlQ+zcYH3OFTbHOk277Qgy+/Lp+rnzzps8HNXX4bKtnysTB4Snb+C+Pf+MR\n4DoN3eeaTx6MNyH633PdbnpupizP7JtTdW9fvliWt7bET+xyu07F8Vz3txAjjmHMTkyEaw79BuPG\nnHMug987y/Tcm8FeUxk8uJnTz3fmsl3bOafHBe3ZNvtRJBBvZ/ZgGdooao9SN37pAcz2Eyjj+irX\n40YeyF5HuLeRc84F8B7kQdxx/8Zl3W5M/OTpuF4HrwXS95qwjkQfvnPONdCLb/ZowLjm/kBHIi6t\nyHN06IA8b6nZJGV9VfaPevzxM6pudlLW1hhPm2V2XQH74KRmf4Ce2TjtDuCoTAghhBBCCCGEjDh8\neSeEEEIIIYQQQkaceyqbR3milZyhXC0w0rUY4loOHRTZ+YHF/aodHrOV6YiMdlPkGMs3RLLQaeto\nrIXFA2X52tJV8wvI/3VEKHM2vwtK3puRPo+52XlpF2qJiI4aEamKXxeV5FspH8QcQbvMXHCUOw3F\nRNRE+u0ltMzIyOA+lPS1Og7CShUz+O51jBNq6XvTBnnYeEfL7CK4x/2+RBRm9txBFReYiEXnyc+5\nxMjzst3lc7fWdWRThLL/QJ+/F8LzAXWRkXU24XoUgZHUe9XXdS9RNy7o6DX9c1VRU9YqgpJfO2Zo\nSe6dnK3bJbNu9x+0UmOcK7JUy+AKkK97xoqCz3OO/cnITkOILbOyUIzKwUjNKDLxN+p3oa1jN5oQ\n+WbRc6FXWYd9w94rpUi29g2shMMHRj6KPxaYcQfHstr41ju0s6T2/PN813YotbftLHheKsLRjMP6\n/I3FgD4PQOnmVU1QY0X0YK4cg0jjdLOr2k0ekvXolz77eVV3/OTJsnx6UdaRKytL+hgQixzaWwd2\nNJTGp3ash8+Z6Q8pyuFtBCNYWlD+npgxtg+S98TY3lQMaVFtb0LZfGqsgBnjOG+D71KefuYbIHnv\n93Q/TAu5tn24tplxqfnwntJs6/Hcg74WJ/KDearvVbyxUpbHA72G7UK/eXdNYhRPHD+u2t1/SmLD\nffPsYezdtWu3VF0fouImJsR2Gvd6qt13vv2nZXlnbUfVjU2gJVXOd6urrXQ9+K7ArM/6XTnmZ774\njLsT+Jd3QgghhBBCCCFkxOHLOyGEEEIIIYQQMuJ8bLJ53+xcHPgibQiNpCuCzwv7RSrfampJut5d\nXUsnxkAePzc9XZYHAy1t0FI1ozkC2VkTjhf39TECkPw2Glq62WrBjuBFteTdSoSQDHZTHJKUwjkG\nNXK6uh1wq2S0ew0tQ/7xSgetRNkv5HlIYGdNz1pF+rBbcWzkv32R4qjgBPNseEoJqCVsSQbyIVMX\nwfOHTovMyIQzkBIZlbMr4Pf2UYZqrRxqB117rdhfndM7/VqJbxjW7DoLHSDHMchZWSjek+ocBbUb\nuLFAqZ3oh/Znhu9GVbM53zzH/qTlbXjOnpFShiCjV8c0csIC5ylzHVOUY+JO3kNjbXV/ZXe9De7q\n71s7DZaHElUglQUtRaZdnldf6Lr5Tx9D6tJUz/O+n0J5d3m6/WztLEhokxMq7Gt183Ndcgw+27nR\nO+v1kpXv0/bxATpJxvQ3XJeF5j6gRB36wE5fy5U3126W5U+e1klIU9Ad5iZlHfnm8nXVLoZjdox1\nKEnhXmJKgU3CyaGuxjaR5nqMzbHvQFcvTHoMbgieD/msdk9Jso8zjuF2jrAWmr2KB3PxynVtr7i0\nIzL0nU0tBZ+anCjL83PS1xpNs3aEIas5pi3IHiR3FZAW5Ae6T4Yw/9p+srEhFswWyN+9TEv0s0R+\nz0Zb9wV8P7t25YaqW9h/WL4bbabdbdXu+y9/ryyfe/+CqvvcF/5CWe6MS4LY6rp+toNIrkFnQr+7\nhh2dlHIn8C/vhBBCCCGEEELIiMOXd0IIIYQQQgghZMThyzshhBBCCCGEEDLi3FPPewixUDYyCj9b\nv3qrKf6Gw4ckKq7T0b4H9HuhL9w55+Jk99iVwrMeDm/Xds4558H/dTQgAi43fqECPEITE5OqbmoS\nPnv6HFXMBvqQ7TnC1w176DB6Rjx51k+nfs5Yjgax9vbtVXS81o/Z826jE6GvDeDeZZF+bpqhPANh\non1ecUM8QjFEgfjGY4b2PPRIOedcMpDvS3e0L8rbEj9S0ZXvHjeRIVkHnu2hmDKILAQfpvXex3m1\nt9Wr8bbuJcIQPGFDPvHq3KwwwjgZKfrGr47HrPPuepUfzDfXPFM4Fobmd0lgA4c802MVHtL25RT2\nrfACvFbmuyHiqDC/gPUzl+1q9w2x0WH8P3PnnOtBHA/2XYvta/i5qnz7M+7zUB3fVjfOq7WCjZsr\ndm9nPfQYKWjXFHVe+aKinf096zz1jQasU9RX1+xFMRQry/H1A9AXWwTVe3bk5hoWcF8yGIdiE6G2\nsrJcliMz/hbgw41h0j4Ka2LnnGs1tJ8W8dCkjP3X9FlP9Xuz/oSOZL3yscP9mHCdrc8D97QpEjN2\nVpXtNVVl22cZFeecc1ur0mcuX9R7I/S2ZQMiex9DmDsbvsyxnXG9/mzBO9jG2rr+cohcHZ+YkeMF\nun+m0Pc8sw9YCvv4NMZhXWn2NLhy8XxZPnh4n6o7d0GeqYvnLqm640dkX4lkINcj7unfZaYj6+eZ\nGR1nt7UtbZtTs2V5Yl63C0P53boDG81YPf9VwVUEIYQQQgghhBAy4vDlnRBCCCGEEEIIGXHuqWwe\nZfVffHoAAA+USURBVGzDcSogKyq05GVqcqwsHz58oCx7zkoVQX7hWbkjysLk35Vk1DmXgxzDxk1g\nhF0AMr9Wp6PaBRBp8MzTz6q6Zz8ln1/83iuqLgGpT1CItCQKquWlhZGG+hifgfFm9r9p4LOV+aGc\nkdwbrKwsQ/kv/HuQW7mNfO6F5iCRxH24seooihS+3MaDeblIf8JY57x5OyLJ6q1vluVBV/efAH6u\n0dDPWwTPvQ8yxNzpMSD34dm2km3K5oeokwnbsVfJhmsiEfVYY9vBZ2hno1+8AKXM9vjQDu+xGeMC\n+K7cjI3JQKwcgW9klhC/6KFs2F4ruAaFkQyr6K2sOmIM5zMbtcUYo9s0mxIZZCXpKDW38xP2V5TS\nDq8pIHqyRmpedTxbZ+0OdbL8Koakv0W11Br7SZ30vk72r9ZcsGbx/aiynbXLYF/e62QYJTwUdSnk\nRbUlMoN7aa/sDljTCjO+TEPEMd4iXG8651wENtPCxid6KIevO1+0oOrxCucBO/Pis4rr2cLMK7XX\nquoa1z1jpi7lGOucc+4HL71ZluO+jvVrQBRlaCwgy0syj66vyrWNmsZuA/Ovb96lpmbmy/LsnAdl\nM75A/w3MOB2C9TMIIVrU6TVmM5K+1t3eUnXn3jsHp6tj6iKQ8OPY2TZRjw8cFNn/yuamqnvnrffL\n8vShR6W8f1G1CyM5/owZU7e2797mwVGZEEIIIYQQQggZcfjyTgghhBBCCCGEjDh8eSeEEEIIIYQQ\nQkace+x5v7P/K7Aeq4WF/WV5ZkZ8P8NewmpPpo3H+oCGOSf0ZM5Oz6i6sY5475sQx/G5zz2n2j36\nyMNl+cjBw6ru/fck0iBNtAfFh/i5uuiZELwqkfE79fviVUGfnPUD4s/VHZ/cG4a8YxAjiNZd63lH\nT1tuY47wGYDiUPKPSg00xwBvZBHpCLhwTJ4HD/x43pb2HLU25TP2T+ecG4AfPnToaTI+Z3xMjX8u\nd9Xe470EPrd+jee9jjr/bO0h1HhV7d9SfvKh/zuWY2CKZpbpcRL3JSnMFBaE4Gkz3UL37WqfM+Y2\n2lQ37YGv3mvBHlId/sccO/n/F7QnvXp+stcL56s6bzx6dRMz1+rzuLNIOXuOIfgi62IU6+431kWh\nPn6jgXN09XqggLXN8DXAayXXYDh+FvbKGfLN8288JXAtPLNOLWrm2Bj6H1q8bURX4qRdZOKUMU5R\nz3kmrg0iY+1aWnWPmjVm3V4M2BczG8dZ4XnPC3uOMu9nebXfXv2WZt5X+6eY9W1eMzbvJdqTsh/X\nzKzuC2hfz80cG6dyfwq4tom5rnFf2jWd8ZPDGD4YiEf93XfeVu3CqF2WF7Z1HHGzI9/d9uX4/gG9\nz9jkjOzvdPWG3ptp9uCpstxN9PvSu9dX5DzacvypuQXVLnUS8b29qWPkVgcSRXfzqsTxHbv/QdXO\ntXDPNF21sb3i7haOyoQQQgghhBBCyIjDl3dCCCGEEEIIIWTEGRnZvIqtMBEZi4siYWi3Rbpr1Wgq\nXchGqKHEBuQ3jWbDVTE5MaE/j02V5f3z+8ryWEvLRSbGRQaysaElFufOSWyB1U54FVJOK+tsteQa\nTE5OqrrlZZFwoBTKxn1gnZXJUzZ/77HCSrR5oMLRuj8KEJZ5Vv07dNQKisoPteeoJG2RPAPejH4e\nBlMSGeInA1Xn90Umlfak7A26+ssgAiwfaOm9M8fcq+QF2A48/QyraB4bvWUkh3eG7g1ZLrI7HF/9\nUMvU1BGG5OoYcyl1fmFkyBC54nJ97kUAUs3ExA3C7x2A9NP3gsp21pqlvkudujlHPP7QvEdJp3PO\nxWCZqYsvrJOhozTTzpMoy6+T1N/pPGmj4hLoX3VRjDif2rqq77LHxOctsJpLjE40VpEQ4kM96OeF\nGR9SFZGbmrpqy8FeI4XxwEqzUTLu+6a/5WgJk34UF1riizFyLtJr0wCiFdE9Z4dRP/Qr6/Az9nW7\nXq6zbWK0m5XUq/UJPLc4Nzmnr5W9jhgrl4P/ILHRnOgKNJGh7XbbEecWDkhcWdbf1pWprKN2TFRZ\nEErfCxrS7zLjRWvA/D6/b07VHT5ysCzje9utW6uq3SCWY/Z7um51VeT2M+PH5ZwCLZvH957rRjbv\n2nBeTd0vojF5x7u5sVGWvZZ+99t0ErW8Fut3up2uXNf335b470fOnlbtAjjGhRV9L5aXzXr3DuBf\n3gkhhBBCCCGEkBGHL++EEEIIIYQQQsiI87HJ5odlcrBba6RP68BBkV+gBA13mrXHH941Vr7P7sBZ\ndR5BqNu1YDfCmRmRYmxsaBnvH//hd8tyo6F/lwikUJ7ZvRalVV4G322u1fa2SC42QOpx+/x3lxta\n6RPWWakg+fixEvgPuDvB7Y8uz1VP0R0eLjPPXq8Jz1RD71jf6MgzFRWyY31hpJoZSLy8HS0x8tY3\n7+zEfsIpauTeQQC7d1sZcoW9wsqQcWwZ2qEdd4r3UZ5rt2tHm4dNCcDkBLANmY6H5+v79txlvC18\nfQ3UHIO/S261pdU7jOP/dyurl5WWwmcrSaUt6TaDgdhd7Hqgfq2wu0Tdzvl181rV8et2hrddPrMy\n3j/H9gX8PeueqcDsNo/nhf3Q9km9U7zdXXz3dAT7TOF32fVRUbC/fkAG9y+1HgW127yRicN6Dm1A\n4dD9gnQA03/70I88GEM831og4Lxsf3Nyn7Fr1Mrm7W7z0HZop3icB1C+n+jfRfVK88jhMRNY4w/M\nnJbCeYQmdekzn9UJUHuVtSWRePe21lRd4Mn1bHe0DB2f+TyHso1wgX7eNGu7+XmxSzZhp/V2R7dL\nEujzxiJ386acfyMS2fnOpm4X+bJ2nBzvqboY3+la2ooyCb/3zpZI9kNzHk89K/3p2srvqLr+5pKc\n76qU337jNdWuMyXpZZvGppB9COcn//JOCCGEEEIIIYSMOHx5J4QQQgghhBBCRhy+vBNCCCGEEEII\nISPOx+Z5H/Z+yed2W/svZmbEK4AeHutPq/e8716uOw/rQYsgqsMHz1GroX0U+N3WFpVDnWfMPujl\nxGPU+fAsVT8XGU9Q3TGtJ5aQu8Gzfkr005k+n8DnFPx4hfEFZ7BXRKOpYzyaEzqiZK+CV33IJ4zl\nobFx97Fg+N9xbLFVGPMG311YryN4ze0hcM8PHIOGYu/AT258oT4c1XqPVbQQfrn1zatcvQ+7b4Qc\nM8v0MYb3C9iboLfaXpMkkT0v7N421Z53PW/V+earjlf3bNjj45xaH4Nb3SczFdGm+3KcQORRgb+z\nnsv172bXROhlb2CFPkn1e1fvZ7HXQb93aiL10Hue2HEJPLQNGM9sv/Fr9hXB/oHxnna+1VHLdn8E\nKeth7kN63u2+JWqhjT9jfhcYw20dxvElcI3tNcWlRGzGiLfffscR525cvlqWQ19fI4y0HtqnI5Z9\nhQLca8DMlTiGh6F+DwpD2M8Iyl5bt+v7Ms7lsd7rCLtTHMt5bG/qOLh+V+q2zB5IRSTHbJlX3nfe\nFI867jk2MPsDHJiVa3X86IKqu3xe+tomJN298uJ3Vbv7Tp4qy0lX79uUxXcfx8m3NEIIIYQQQggh\nZMThyzshhBBCCCGEEDLi3FPZPMpjrOoSZTpTk5Oqbm5OZLE+SsSMxE3JgGqi0eqokuQ551wY7h4p\nE5joFvxdwrA6ashKT6tk//Z3qatDmiDzt/FEGF9jr01YIzEk5IdhY+4aKf5DtRQ7r4gsc845H6Ie\nbY/vRuyvzjkX1EVD1kiDMSbIq/n/3Dr3TtXYPhQH56Gk3sgxU5GOeSFG2xnpZwDRNZ6uy3ORBoYm\nugZlp4XS0FdfDysZxqZ1UZwBnKNVyaMkfC+D1284kk+w8mH8jOXMyJjxngzNceHu0mXbTq8B7HlU\nz71I3ZoCP9v4NvycZyi9r74eti5JUqgDqamvpfc+9NcgqJbG7nXw6qYmKjBwKCc3MnG4hilEdNmx\nANdeOJY5p20aETwv1uaIKmdrxVBjvQfnbyZtHXVpfpe8WvKOdf1Y1phpXh0VNyzL373d8Dgg5TCy\ncui3HHFufk4sx1OTbVXX622V5UZbjwcFdEs1xRrZPPbJdntc1fV7cs97vZ2yPPQuEqe7lv/828vS\njaUbcuwtHc99+cKVsvzmO2+rus9+4Yvy3Z7uhyE8A52OnP/2zRuqXf+WRNbdN6+v45H9YvNeXpdr\nunpzWbWbXzhalu08kPr29/7hcFQmhBBCCCGEEEJGHL68E0IIIYQQQgghIw5f3gkhhBBCCCGEkBHn\nnnreC+UTN/9vAP6hhf37VdUkeOAxHiIf8iOCr7PGu+bVxHGo41l3Lfh2AvxdTIRMBqYdGy+D32d9\nflVxNvYcMTqnNk6kJioOPU3Dfk16iMlHCfqkTCRNRSvrL/XRB2c8yh86zesnjFwbBFUdPtGZidXB\nfURw2LQeXBwmbDyRj77YotocXxVVdPvH4DwgTsn6eG3Apv4IU5rxcQYhxs+BX702uk0fH8d2fT2M\n79SBv94ev+b67CX0vFMd7WrnyapoN9tPEHsP8LvjWLzgQ9Fdah8aM1/DeflBzX4dPp6vPr79rM4Z\nj4FRjIHdo0HKUWT335Ey9t2ssGsW6a+x9Z0y2lBIMTbLxP7BM+/XRPNivGURGR833Fobw4ZDLI4v\nQ/tH4fNh9yvAZ0T9oO2HsK/E0F4Pux/u9mdcg8O+H2YpraPo7DFwbK7ec8KHXyC0MY41++fsJfq4\nD8hOT9UNBjLujRkfegFjVhjg3i76+DhPJ4mujKGft9uy/5ZJunStSDzjUVPfRxyLlq5fL8ubt/QY\n9f55iHwz8+uBxfmyPD6uffkTLYmt8zzZI2flXbMvTSqe947bUVUnD8n76da2xNTl6UC1m5yW99qJ\n2cOqLk60F/9O4F/eCSGEEEIIIYSQEYcv74QQQgghhBBCyIjj1cnGCSGEEEIIIYQQ8vHDv7wTQggh\nhBBCCCEjDl/eCSGEEEIIIYSQEYcv74QQQgghhBBCyIjDl3dCCCGEEEIIIWTE4cs7IYQQQgghhBAy\n4vDlnRBCCCGEEEIIGXH48k4IIYQQQgghhIw4fHknhBBCCCGEEEJGHL68E0IIIYQQQgghIw5f3gkh\nhBBCCCGEkBGHL++EEEIIIYQQQsiIw5d3QgghhBBCCCFkxOHLOyGEEEIIIYQQMuLw5Z0QQgghhBBC\nCBlx+PJOCCGEEEIIIYSMOHx5J4QQQgghhBBCRhy+vBNCCCGEEEIIISMOX94JIYQQQgghhJARhy/v\nhBBCCCGEEELIiMOXd0IIIYQQQgghZMThyzshhBBCCCGEEDLi8OWdEEIIIYQQQggZcfjyTgghhBBC\nCCGEjDj/Dd78Z8UCB1WgAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f22afd64d68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "_, data_testing = myutils.load_CIFAR_dataset(shuffle=False)\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline \n",
    "\n",
    "fig = plt.figure(figsize=(18,2));\n",
    "for _i in range(conf_matrix[i,j]):\n",
    "    a=fig.add_subplot(1,conf_matrix[i,j],_i+1)\n",
    "    plt.imshow(data_testing[img_idx[_i]][0])\n",
    "    plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Saving parameters\n",
    "We simply save the matrix with weights and bias vector for linear classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# np.savez_compressed(\"classifiers/9158_resnet50-keras_LinearSVC.npz\",W=np.array(clf.coef_).T, b=clf.intercept_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## k-nearest neighbors classifier\n",
    "\n",
    "Let us note that simple [kNN classifier](http://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html#sklearn.neighbors.KNeighborsClassifier)\n",
    "(with k=10), trained with 5000 training features (CNN codes from Inception_v3) gives **83.45%** accuracy on whole 10000 testing images.\n",
    "\n",
    "Remark that computing predictions with this classifier is very complex and it is not recommended for classificcation of images.\n",
    "\n",
    "Here is the code to compute the score on testing dataset.\n",
    "```python\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "kNN_clf = KNeighborsClassifier(n_neighbors=10)\n",
    "kNN_clf.fit(X_training, y_training)\n",
    "print( 'Classification score = ', kNN_clf.score( X_testing, y_testing ) )\n",
    "# Classification score =  0.8345\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we used <tt>[Logistic regression](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression)</tt> with default parameters. We trained the model with all the training data and obtained **90.37%** accuracy on testing dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear regression accuracy =  0.9037\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "clf = LogisticRegression()\n",
    "clf.fit(X_training, y_training)\n",
    "print( 'Linear regression accuracy = ', clf.score( X_testing, y_testing ) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
